Part Ii Memory , Emotion , Aging , and the Brain

Emotional events are usually remembered better than neutral events. The anatomical and functional correlates of this phenomenon have been investigated in both animals and humans, with approaches ranging from neuropsychological and pharmacological to electrophysiological and functional neuroimaging. The present chapter reviews this evidence, focusing in particular on functional neuroimaging studies in humans, which have examined the effects of emotion on memory-related activity during both encoding and retrieval. The available evidence emphasizes the role of the amygdala, the medial temporal lobe memory system, and the prefrontal cortex. The chapter ends with a discussion of open issues and future directions. Among the multitude of stimuli comprising our environment, at any moment there are some stimuli that are more relevant to us than others, and thus are more likely to engage our processing resources. What is it, though, that makes some stimuli more relevant than others? To some extent, stimuli with virtually identical properties may still be preferentially processed because they better serve our current activities and goals. Nevertheless, some stimuli tend to “capture” our attention regardless of their relevance with respect to our present activity. This latter category comprises emotional stimuli, which as a result of evolution and ontogenesis have gained relevance that transcends that of present activities. What are the neural mechanisms though that allow for such privileged access to processing resources that can make us both emotionally resonate and behave accordingly by approaching something that is desirable, or avoiding something that is potentially harmful? Also, what are the mechanisms that make possible the transition from the activity-specific to the more general relevance of these stimuli? Particularly, what are the mechanisms underlying our ability to encode, store, and retrieve information about our experiences, which ultimately allow 6 The Memory Enhancing Effect of Emotion: Functional Neuroimaging Evidence 108 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza us to distinguish what may be approached from what should be avoided? Finally, how do alterations of these mechanisms contribute to clinical conditions, such as apathy, anxiety, depression, or post-traumatic stress disorders? These are only a few questions that humans have tried to answer for centuries, and neuroscientists have fervidly engaged in answering particularly during the last few decades. Consequently, a new field has emerged: the cognitive neuroscience of emotion. Different approaches ranging from neuropsychological and pharmacological to functional neuroimaging have attempted to define the neural mechanisms underlying the operations associated with different aspects of emotional processing in both humans and animals (reviewed in Damasio, 1994; Davidson & Irwin, 1999; Davis & Whalen, 2001; Lane & Nadel, 2000; LeDoux, 2000; McGaugh, 2004; Phan et al., 2002; Phelps, 2004; Zald, 2003). This chapter focuses on evidence concerning the neural correlates of the memory-enhancing effect of emotion, as obtained from functional neuroimaging studies of neurologically intact human subjects. In the following sections, we first introduce a few key concepts, and then highlight the role of functional neuroimaging techniques in revealing the neural correlates of the modulatory effect of emotion during encoding and retrieval of episodic memories (i.e., explicit memory for personal events; Tulving, 1983). The chapter ends with a discussion of future directions. CONCEPTS AND METHODS Recent developments in the cognitive neuroscience of emotion have led to important progress in identifying the nature of emotional phenomena and their relationship with other psychological processes, as well as in understanding their features and neural correlates. Among the various psychological phenomena involving emotional processing in humans, the modulatory effect of emotion on different stages of memory (e.g., encoding, retrieval) has received substantial attention in the literature. Concerning the main features defining emotional phenomena, a critical distinction in the emotion literature is the one between two orthogonal affective dimensions: arousal and valence (Lang et al., 1993; Russell, 1980). Arousal refers to a continuum that varies from calm to excitement, whereas valence refers to a continuum that varies from pleasant to unpleasant with neutral as an intermediate value (for methods to assess these dimensions, see Bradley & Lang, 1994). Specific emotions can be positioned within this dimensional space according to the specific arousal and valence levels associated with them. In addition, important progress has also been made regarding the neural mechanisms underlying the operations associated with various aspects of emotional processing. Among the brain regions most typically associated with emotional processing, the amygdala and the prefrontal cortices have been the most thoroughly investigated (e.g., Damasio, 1994; Davidson & Irwin, 1999; Functional Neuroimaging of Emotional Memory 109 Davis & Whalen, 2001; Lane & Nadel, 2000; LeDoux, 1996; McGaugh, 2000; Phan et al., 2002; Phelps, 2004; Zald, 2003), although the role of other brain regions such as the basal ganglia and insular cortex has also been studied (e.g., Davidson & Irwin, 1999). A significant contribution to this recent progress has been made by functional neuroimaging methods. Particularly important are non-invasive techniques, such as those involving hemodynamic (e.g., event-related functional magnetic resonance imaging: ER-fMRI) or electrophysiological (e.g., event-related potentials: ERP) measurements, both of which are assumed to be associated with changes in the neural activity underlying various psychological processes. Functional MRI typically involves measurements of blood oxygenation, and is characterized by excellent spatial resolution (at the level of millimeters), whereas ERPs typically involve recording of electrical potentials through electrodes placed on the scalp, and are characterized by excellent temporal resolution (at the level of milliseconds). These two techniques can thus offer complementary information concerning the brain mechanisms and the timing associated with various cognitive operations. An important advance in understanding the neural correlates of emotional memory has been made by application of event-related designs and the subsequent memory paradigm. One important feature of event-related designs is that recording of functional neuroimaging data (e.g., ER-fMRI/ERP) can be time-locked to “events” that occur either in the external environment (e.g., the onset of a stimulus) or in the participants’ minds (Picton, Lins, & Scherg, 1995; Rugg, 1995). In contrast to other functional neuroimaging techniques that use “blocked” designs (e.g., blocked fMRI and Positron Emission Tomography – PET), ER-fMRI and ERP data can be analyzed offline on a trial-by-trial basis and selectively sorted and averaged on the basis of subjects’ behavior. For instance, one may compare activity for items that are remembered vs. forgotten in a memory test, thereby establishing a direct link between brain activity and successful memory performance in individual participants. Comparing brain activity for remembered vs. forgotten items can be done both during the encoding (learning) phase and during the retrieval (test) phase. During encoding, one can identify regions showing greater activity for items that are subsequently remembered than for items that are subsequently forgotten in a later memory test (subsequent memory paradigm). The difference in activity between remembered vs. forgotten items during encoding is known as difference in memory or Dm effect (e.g., Paller, Kutas, & Mayes, 1987; Paller & Wagner, 2002). Brain regions showing Dm effects are assumed to mediate processes that lead to successful encoding of incoming information. The “remembered-minus-forgotten” difference can be also investigated during retrieval. In this case, the contrast identifies regions showing greater activity for items in which retrieval was successful (hits) than for items in which retrieval failed (misses). The activity difference between remembered vs. forgotten 110 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza items during retrieval is known as retrieval success (Prince, Daselaar, & Cabeza, 2005; Weis et al., 2004). One of the advantages of Dm and retrieval success analyses is that they can disentangle the effects of emotion on perception-related vs. memory-related activity. The effects of emotion on perception-related activity can be identified by comparing activity for processing emotional vs. non-emotional stimuli (for reviews, see Davidson & Irwin, 1999; Phan et al., 2002; Wager et al., 2003). In contrast, identifying the effects of emotion on memory-related activity requires isolating first activity specifically associated with memory processes, that is, successful encoding activity (Dm effect) and successful retrieval activity. For example, in a study of emotional memory encoding, one can identify the brain regions associated with encoding success, separately for emotional (emotional Dm = remembered emotional stimuli – forgotten emotional stimuli) and neutral stimuli (neutral Dm = remembered neutral stimuli – forgotten neutral stimuli). By contrasting the emotional Dm to the neutral Dm, the effects of emotion on successful memory activity can be identified. Similarly, during retrieval, one can identify the brain regions where successful retrieval activity is greater for emotional than for the neutral stimuli. In combination, these comparisons reveal brain regions that show an interaction between emotion and memory at different stages of memory processing. In the next sections, we will review evidence from functional neuroimaging studies investigating the neural correlates of the modulatory effect of emotional arousal on episodic memory encoding and retrieval. The emphasis will be on the role of the amygdala and the medial temporal lobe (MTL) memory regions, including the hippocampus and parahippocampal regions (Squire & Zola-Morgan, 1991). In addition, we will emphasize ER-fMRI studies that allow identification of brain regions showing greater encoding/ retrieval success activity for emotional than for neutral events. FUNCTIONAL NEUROIMAGING OF EMOTIONAL MEMORY Neural mechanisms of emotional memory encoding Role of the amygdala and the MTL memory regions One influential hypothesis concerning the neural mechanisms underlying the beneficial effect of emotion on memory (Bradley et al., 1992; Christianson, 1992b) is the so-called modulation hypothesis (McGaugh, Cahill, & Roozendaal, 1996; McGaugh, McIntyre, & Power, 2002). According to this hypothesis, Functional Neuroimaging of Emotional Memory 111 emotional events are remembered better than neutral events because the amygdala enhances the function of MTL memory structures associated with declarative memory and other brain regions involved in non-declarative memory (McGaugh, 2000). Studies on non-human animals have clearly established that the amygdala, a brain region important in the processing of emotional arousal, is critical for mediating the influences of both neuronal and neurohormonal mechanisms that lead to enhanced memory for emotional events (McGaugh, 2000; McGaugh et al., 1996, 2002). Moreover, these studies also specifically have identified the basolateral nucleus of the amygdala (BLA) as the main site responsible for the modulatory influences of the amygdala on activity in brain regions associated with mnemonic processes. Importantly, at a basic level, the neuronal and neurohormonal mechanisms mentioned above involve activation of the central and peripheral adrenergic systems, which are engaged as a result of experiencing emotionally arousing events, as part of the general response to stressful situations. Pharmacological, lesion, and functional neuroimaging studies in humans also provide evidence supporting the modulation hypothesis (for reviews, see Cahill & McGaugh, 1998; Hamann, 2001; McGaugh, 2004; Phelps, 2004). Most of these studies have presented participants with visual images (words or pictures) or audiovisual narratives that vary in their emotional content, and assess memory performance as a function of stimulus category. Pharmacological studies have shown that adrenergic agonists given prior to or after a learning episode enhanced memory performance (Soetens et al., 1995), whereas adrenergic antagonists specifically decreased memory performance for emotional information (Cahill et al., 1994). Lesion studies show that patients with amygdalar lesions often do not exhibit a memory advantage for emotionally arousing stimuli (Adolphs et al., 1997; Adolphs, Tranel, & Denburg, 2000; Cahill et al., 1995; LaBar & Phelps, 1998; Phelps et al., 1998). Finally, functional neuroimaging studies have yielded some key findings that provide complementary support for the modulation hypothesis: (1) activity in the amygdala correlates with memory for emotionally arousing stimuli (e.g., Cahill et al., 1996); (2) this effect holds for both pleasant and unpleasant stimuli (Hamann et al., 1999); (3) memory-related activity in the amygdala correlates with the intensity of the emotional response (Canli et al., 2000); (4) increased memory performance for emotional stimuli is associated with successful encoding activity in both the amygdala and the MTL memory regions (Dolcos, LaBar, & Cabeza, 2004b; Kensinger & Corkin, 2004), which is greater for emotional than for neutral stimuli (Dolcos et al., 2004b); and (5) the memory-enhancing effect of emotional arousal is associated with interaction between the amygdala and the MTL memory regions (Dolcos et al., 2004b; Kensinger & Corkin, 2004; Kilpatrick & Cahill, 2003; Richardson, Strange, & Dolan, 2004). 112 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza Despite their consistency, most evidence from studies on humans has been rather indirect and/or lacks spatial specificity with respect to the neural mechanisms responsible for the memory-enhancing effect of emotion. Pharmacological studies in neurologically intact humans, for instance, can only indirectly link the modulatory effect of adrenergic agonists/antagonists to activity in the amygdala or other brain regions. As for the evidence from neuropsychological studies, because lesions are typically not restricted to the amygdala proper, it is unclear whether the deficits in memory performance for emotional items reflect a lack of amygdalar modulation or damage of the neighboring MTL regions. Similar spatial limitations also apply to most functional neuroimaging studies. Below, we will review the main limitations of the functional neuroimaging approach, while presenting the chronological development that cumulatively provides strong evidence for the modulation hypothesis in neurologically intact human participants. The main limitations of the early PET and fMRI studies of emotional memory encoding are related to the nature of blocked-trial designs, which cannot accommodate the subsequent memory paradigm. Early PET studies suggested a link between amygdala activity at encoding and later retrieval of emotionally arousing material (Cahill et al., 1996; Hamann et al., 1999). However, despite their better spatial resolution relative to pharmacological and lesion studies, these studies could not distinguish brain activity for successfully vs. unsuccessfully encoded stimuli within participants, and thus could not directly assess the role of the amygdala in successful encoding of emotional stimuli. More recent fMRI studies (e.g., Canli et al., 2000) have used event-related designs to examine emotional memory formation, but they did not employ the subsequent memory paradigm either. Furthermore, these studies have focused mainly on the amygdala, and thus did not provide evidence for the interaction of the amygdala with other memory-related brain regions during the formation of emotional memories. An ERP study by Dolcos and Cabeza (2002) employed the subsequent memory paradigm with emotional stimuli and reported greater successful encoding activity (Dm) for emotional than for neutral stimuli. Event-related potentials were recorded while subjects rated a randomized series of emotionally arousing (both pleasant and unpleasant) and neutral pictures for pleasantness. Based on memory performance in a subsequent memory task, ERPs were separately averaged for each emotion category and, within each category, for subsequently remembered and forgotten items. The main finding was that enhanced memory performance for emotional pictures was associated with greater successful encoding activity (Dm) for emotional than for neutral items during an early time window (i.e., 400–600 ms). This result suggests that emotional stimuli have privileged access to processing resources, which could be one of the mechanisms responsible for the memory-enhancing effect of Functional Neuroimaging of Emotional Memory 113 emotion. Given the relatively closed-field configuration of amygdalar neurons, which yields little volume conduction (Gloor, 1992), it is unlikely that the amygdala was a direct source of this scalp ERP. It is possible, however, that the amygdala may influence such ERPs measured at the scalp indirectly through its widespread cortical projections. Recent studies have made significant advances by incorporating the subsequent memory paradigm and by considering how the amygdala interfaces with other memory-processing areas of the brain (Dolcos et al., 2004b; Kensinger & Corkin, 2004; Kilpatrick & Cahill, 2003; Richardson et al., 2004). For instance, the study by Kilpatrick and Cahill (2003) involved structural equation modeling with PET data and found that increased memory for subsequently remembered emotional vs. neutral video clips was associated with increased functional connectivity between the amygdala and parahippocampal regions. The study by Richardson and colleagues (2004) involved ER-fMRI recording in patients with amygdala and hippocampal lesions and found evidence for reciprocal dependence between these two brain regions during the encoding of emotional memories: hippocampal activity for subsequently remembered emotional verbal material correlated with the degree of amygdalar pathology, and amygdalar activity for the same items correlated with the degree of hippocampal pathology. Although these studies did not involve the subsequent memory paradigm to compare activity for subsequently remembered vs. forgotten items, they provide support for the idea that the amygdala exerts modulatory influences on encoding activity in memoryrelated brain regions (see also Hamann et al., 1999). Finally, the ER-fMRI studies by Dolcos et al. (2004b) and Kensinger and Corkin (2004) involved event-related designs and the subsequent memory paradigm and found that the memory-enhancing effect of emotion was associated with greater successful encoding activity for emotional than for neutral stimuli (emotional Dm > neutral Dm) in both the amygdala and the MTL memory regions, as well as with greater amygdala–MTL correlations. In sum, these studies have provided strong evidence supporting the notion that the memory-enhancing effect of emotional arousal involves interactions between the amygdala and the MTL memory structures during the initial encoding of emotionally charged events. It is important to note that, despite the advances in recent ER-fMRI studies, there are still limitations in the spatial resolution of these studies. Specifically, most of the studies reviewed above have used voxel-wise analyses, which do not provide the most accurate localization of activity coming from neighboring MTL regions because of image smoothing and normalization of individual variation in human anatomy to a standard brain template. This limitation is illustrated in figure 6.1, which shows the results of a voxel-based analysis performed on normalized and smoothed fMRI data comparing activity for remembered emotional and neutral pictures from an event-related fMRI study of emotional memory encoding (Dolcos et al., 2003). As indicated by the 114 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza color-coded lines delineating the amygdala–hippocampal border, the activation encompasses portions of both regions (i.e., the amygdala and hippocampus head), but due to normalization and smoothing it is difficult to precisely separate activity from these two neighboring areas. In addition, there is individual variation in MR-related susceptibility artifact in the vicinity of the amygdala that can reduce overall activation in voxel-based group-averaged analyses due to susceptibility-related offsets in spatial registration of observed activity (LaBar et al., 2001). These issues were addressed in a recent ER-fMRI study (Dolcos et al., 2004b) that used the subsequent memory paradigm with emotional and neutral stimuli similar to that used in the ERP study described above (Dolcos & Cabeza, 2002). In this study, data analysis entailed extraction of the fMRI signal from anatomically defined regions of interest (ROIs) within adjacent MTL structures that were manually traced on each subject’s anatomical scans (figure 6.2A–B). In addition, preprocessing of fMRI data did not involve normalization or Figure 6.1 Illustration of the difficulty to separate the fMRI signal coming from two MTL neighboring regions (i.e., the amygdala and the hippocampus head), if smoothing and normalization to a standard template is involved. The figure shows a sagittal view of a high-resolution brain image with an activation t map superimposed. Although the findings (greater amygdalar-hippocampal activity for remembered-emotional than for remembered-neutral stimuli) are interesting and consistent with the modulation hypothesis, it is difficult to delineate the contribution of the two neighboring regions to the observed effect. Amy = amygdala, HC = hippocampus. The colored lines follow the sagittal borders of the two regions. Functional Neuroimaging of Emotional Memory 115 Figure 6.2 Greater emotional Dm than neutral Dm in the amygdala and MTL memory system. (A) Three-dimensional view of the anatomically defined ROIs from one representative subject. (B) Coronal view of eight representative slices showing the location in the brain of the MTL subregions. (C) Bar graphs comparing the percent signal change for emotional and neutral Dms, extracted from the MTL subregions showing greater emotional than neutral Dm. The bar for emotional Dm is based on the fMRI signal averaged across the emotional conditions (pleasant and unpleasant collapsed). Ctx. = Cortex, Dm = Remembered – Forgotten. 116 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza smoothing. This analysis method enabled a more precise quantification of signal changes arising from subregions of the amygdala, and the MTL memory regions (including the hippocampus and the parahippocampal cortices: entorhinal cortex, perirhinal cortex, and parahippocampal cortex proper). Results showed that memory performance was greater for emotional (both pleasant and unpleasant) than for neutral stimuli. This emotional memory advantage was reflected in greater emotional Dm activity than neutral Dm activity in both the amygdala (especially the BLA) and the MTL memory system (especially entorhinal cortex and the head of the hippocampus; figure 6.2C). Moreover, Dm activity in the amygdala and the MTL memory regions was more strongly correlated with each other for the emotional stimuli than for the neutral stimuli (figure 6.3). These findings reveal interactions between an amygdala-based emotional processing system and other memory-related MTL regions that support successful encoding of emotional stimuli. Finally, a double dissociation was found along the longitudinal axis of the MTL memory system such that activity in anterior regions predicted memory for emotional items whereas activity in posterior regions predicted memory for neutral items. These anterior portions of the hippocampus and parahippocampal gyrus are directly and reciprocally interconnected with the amygdala (Amaral et al., 1992) and may form a mechanism by which emotion and memory are synergistically enhanced. In sum, these results complement the studies reviewed above to provide key evidence in support of the modulation hypothesis in humans, and they also uncover a functional specialization along the rostrocaudal axis of the MTL regarding the effects of emotion on memory formation. Figure 6.3 Stronger correlations between successful encoding activity in the amygdala and the entorhinal cortex for emotional Dm (left panel) than for neutral Dm (right panel). The plots are based on the emotional and neutral Dms, as extracted from left amygdala and left entorhinal cortex (Ctx.). Dm = Remembered – Forgotten, L = Left. Functional Neuroimaging of Emotional Memory 117 Role of the prefrontal cortical regions (PFC) Although the findings discussed above link the memory-enhancing effect of emotion to an MTL mechanism, as predicted by the modulation hypothesis, they do not exclude the possibility that other brain regions, such as the PFC, also play a major role. Functional neuroimaging studies have consistently shown that PFC regions are as strongly associated with successful encoding operations as are MTL regions (e.g., Brewer et al., 1998; Paller & Wagner, 2002; Wagner et al., 1998). Thus, it is quite likely that the memory-enhancing effect of emotion is also mediated by the PFC. The role of PFC in memory is diverse, and PFC contributions to strategic, organizational, and semantic aspects of emotional memory may operate in the absence of high arousal (see discussion in Kensinger et al., 2002; LaBar, 2003; Phelps et al., 1998) and can be valence-specific (Dolcos, LaBar, & Cabeza, 2004a; Kensinger & Corkin, 2004). Here we focus on the role of the PFC in arousal-mediated memory effects to complement our discussion of the MTL mechanisms above. Several studies have reported a role of PFC in arousalmediated memory effects (e.g., Canli et al., 2002; Dolcos et al., 2004a; Kilpatrick & Cahill, 2003; Sergerie, Lepage, & Armony, 2005). For instance, the PET study by Kilpatrick and Cahill (2003) also reported increased functional connectivity between the amygdala and the PFC, specifically in a ventrolateral PFC region (Brodmann area 47), during the encoding of highly arousing negative vs. low arousing neutral video clips. Also, the ER-fMRI study by Canli et al. (2002) reported that encoding activity in a dorsolateral PFC region (BA 6) correlated positively with both greater emotional arousal and better subsequent memory for pictorial material. Moreover, Dolcos et al. (2004a) found that emotional arousal enhanced successful encoding activity (Dm) in two subregions of the left lateral PFC: one ventral (BA 47) and the other dorsal (BA 9/6). In these regions, the Dm effect was greater for highly arousing (both pleasant and unpleasant) items relative to low arousing and neutral ones. The ventrolateral PFC regions have been consistently associated with encoding success in previous functional neuroimaging studies of episodic memory (e.g., Brewer et al., 1998; Kirchhoff et al., 2000; Paller & Wagner, 2002; Wagner et al., 1998), thus it is not surprising that they were also identified by the above-mentioned studies of emotional memory encoding. Given the role of left ventrolateral PFC regions in semantic processing (Kapur et al., 1996; Poldrack et al., 1999; Shallice et al., 1994) and the role of the dorsolateral PFC regions in working memory operations (D’Esposito, Postle, & Rypma, 2000; Owen et al., 1999; Petrides, 1995), it is possible that arousing events are better remembered in part because they receive deeper semantic processing and attentional maintenance in working memory buffers during encoding (see also Reisberg, this volume). 118 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza These results expand the above-mentioned evidence concerning the role of the MTL during emotional memory encoding by showing that the enhancing effect of emotion on memory formation is also mediated by changes in PFC activity. The effects of emotion on MTL and PFC regions, however, may relate to different memory mechanisms. Given the functions typically attributed to these regions (Moscovitch, 1992; Simons & Spiers, 2003), it is reasonable to assume that, in MTL, emotion enhances the storage and consolidation of memory representations whereas, in PFC, it enhances strategic encoding processes. Taken together, these findings suggest that the enhancing effect of emotion on memory formation (1) is largely but not exclusively influenced by arousal; (2) is mediated by changes in both MTL and PFC activity; and (3) may involve an enhancement of storage and consolidation processes in MTL, as well as an amplification of semantic and working memory processes in PFC. Neural mechanisms of emotional memory retrieval Effect of emotion on retrieval activity in the amygdala and the MTL memory regions What about the mechanisms involved during retrieval of emotional memories: are they the same as those identified during encoding? Although it is reasonable to expect that emotional arousal can enhance not only early memory stages (i.e., encoding and early consolidation), but also later stages (e.g., retrieval), very little is known about the neural correlates of emotional memory retrieval. The paucity of functional neuroimaging evidence may reflect the fact that the modulation hypothesis as inspired by early animal research focuses mainly on encoding and consolidation (McGaugh, 2004), and thus the majority of studies on the neural mechanisms of emotional memory have correspondingly focused on encoding (e.g., Cahill et al., 1996; Canli et al., 2000, 2002; Dolcos & Cabeza, 2002; Dolcos et al., 2004b; Hamann et al., 1999; Kensinger & Corkin, 2004; Kilpatrick & Cahill, 2003; Richardson et al., 2004; Sergerie, Lepage, & Armony, 2005). Recent evidence from animal research, however, suggests that the amygdala is also involved in reconsolidation processes initiated upon retrieval of emotional memories (LeDoux, 2000), although the exact nature of its involvement is a matter of current debate (Nadel, 2000; Nader, 2003). The situation in the functional neuroimaging literature is very similar – some studies do not report amygdala activity during emotional memory retrieval (e.g., Kosslyn et al., 1996; Taylor et al., 1998), and most of the studies that do report activity in the amygdala could not directly link it to retrieval processes per se (e.g., Dolan et al., 2000), and/or to increased memory for Functional Neuroimaging of Emotional Memory 119 emotional items compared to neutral items (e.g., Fossati et al., 2004; Maratos, Allan, & Rugg, 2000; Sharot, Delgado, & Phelps, 2004). One study reported evidence linking activity in the amygdala with retrieval processes and with emotion-mediated enhancement of memory performance (Smith et al., 2004b), but this study focused on identifying the neural correlates of retrieving neutral items encoded in emotional versus neutral contexts (see also Maratos et al., 2001; Smith et al., 2004a), rather than on identifying the neural correlates of retrieving emotional vs. neutral items. These retrieval studies also share many of the same limitations as the early studies of emotional memory encoding. For instance, most of them either used blocked designs that do not allow assessment of the functional neuroimaging data on a trial-by-trial basis (e.g., Dolan et al., 2000; Kosslyn et al., 1996; Taylor et al., 1998), and/or did not compare activity associated with successfully vs. unsuccessfully retrieved items to distinguish brain activity specifically associated with retrieval success from brain activity generally associated with perceptual processing (e.g., Fossati et al., 2004; Maratos et al., 2000; Sharot et al., 2004). Moreover, as a general limitation, most of these studies have used short retention intervals (e.g., minutes), which could explain why some of them did not find a memory advantage for emotional stimuli. Additionally, the involvement of short retention intervals could not allow a clear separation between early consolidation processes and actual retrieval operations. We addressed these issues in a recent ER-fMRI study (Dolcos, LaBar, & Cabeza, 2005) by creating an experimental design with the following key features: (1) it allowed identification of transient brain activity on a trial-by-trial basis using an event-related design; (2) it compared activity for successfully (hits) vs. unsuccessfully (misses) retrieved items for emotional and neutral items to identify the MTL subregions associated with the interaction of emotion and successful memory retrieval; and (3) it assessed retrieval following a retention interval of one year to distinguish retrieval processes from early consolidation processes. Participants encoded highly arousing emotional (pleasant and unpleasant) and low arousing neutral pictures, and one year later they were scanned while distinguishing between the pictures they had seen previously vs. new pictures. These subjects were a subgroup of the participants that were also scanned during encoding (Dolcos et al., 2004a, 2004b). Finally, activity in the amygdala and the MTL memory regions was extracted using a MTL mask, which can localize activity more precisely from various MTL subregions. We found that, one year after encoding, emotionally arousing stimuli (both pleasant and unpleasant) were remembered better than neutral stimuli, and this behavioral effect was associated with greater retrieval success activity in the amygdala and the MTL memory system for emotional than for neutral items. Importantly, this activation cannot be attributed to general emotional perceptual processes, as it reflected a difference between retrieval activity for 120 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza emotional pictures correctly classified as old (Emotional Hits) and activity for emotional pictures incorrectly classified (Emotional Misses). Moreover, this difference was also identified when emotional retrieval success activity (Emotional Hits > Emotional Misses) was compared to the neutral retrieval success activity (Neutral Hits > Neutral Misses). Therefore, activity related to perception of emotion was subtracted out, and the observed differences reflect the interaction between emotion and memory. These findings provide strong evidence that successful retrieval of emotional memories involves MTL mechanisms similar to those identified during successful emotional encoding. Effect of emotion on recollectionvs. familiarity-based retrieval In addition to the general issue concerning the effect of emotion on the neural correlates underlying memory retrieval, another issue is the specific effect of emotion on the neural mechanisms associated with different types of memory retrieval processes. One such distinction refers to differences between the neural mechanisms associated with recollectionvs. familiarity-based retrieval operations. Memories for personal events are not always retrieved in the same way; some are fully recollected (for instance, we can remember rich details about the time and place that specific events took place), whereas other memories are only familiar to us (we just know that certain events took place, but cannot retrieve specific details about their occurrence) (Gardiner, 1988; Jacoby & Dallas, 1981; Mandler, 1980; Tulving, 1985). The theoretical models proposing such dissociations are not new in the memory literature, but only recently have received support from neuroimaging studies concerning the existence of distinguishable neural mechanisms underlying these two types of retrieval (for a review, see Yonelinas, 2002). Distinguishing recollection from familiarity is critical because there is behavioral evidence that the memory-enhancing effect of emotion specifically modulates recollection rather than familiarity processes (Ochsner, 2000; Talarico, LaBar, & Rubin, 2005). Although several functional neuroimaging studies have identified dissociable neural correlates supporting recollectionvs. familiaritydriven retrieval (Dobbins et al., 2003; Eldridge et al., 2000; Henson et al., 1999), it is not clear how emotion modulates these mechanisms in order to enhance our ability to distinguish between what we remember from what we know. It is possible that this effect relies on arousal-mediated enhancement of activity in brain regions that distinguish between recollection and familiarity (e.g., hippocampus). However, most of the functional neuroimaging studies investigating the neural correlates of recollection and familiarity (e.g., Dobbins et al., 2003; Eldridge et al., 2000; Henson et al., 1999) have not examined the effect of emotion on these two types of memory retrieval. Functional Neuroimaging of Emotional Memory 121 Two recent studies have provided evidence concerning the neural correlates of the differential effect of emotion on recollectionvs. familiarity-based retrieval (Dolcos et al., 2005; Sharot et al., 2004). For instance, the study by Sharot et al. (2004) measured brain activity associated with the feeling of remembering during retrieval of emotional and neutral items, using a recognition task that distinguishes between recollectionand familiarity-based responses (Tulving, 1985). Thus, in addition to distinguishing between previously seen (Old ) and new items, for each item considered as Old participants also indicated whether their memory for the item was accompanied by specific details about its occurrence during encoding session (“Remember-R” responses), or whether they only knew that the item was old but could not retrieve specific details (“Know-K ” responses). Sharot et al. found that remember judgments for emotional items were enhanced, even though the actual retrieval accuracy did not differ for the emotional and neutral categories, and that this enhanced feeling of remembering for emotional items was associated with increased activity in the amygdala. The study by Dolcos et al. (2005) reviewed above provides further evidence for the effect of emotion on recollection vs. familiarity by identifying the neural mechanisms associated not only with enhanced feeling of recollection but also with enhanced actual recollection of emotional memories. The differential effect of emotion on recollection and familiarity was investigated using the same recognition task (Tulving, 1985) that distinguishes between recollectionand familiarity-based responses. To identify the MTL regions associated with the effect of arousal on recollection vs. familiarity, retrieval success activity (Hits – Misses) for emotional and neutral items was calculated separately for recollectionvs. familiarity-based responses. Further, to investigate whether the amygdala and the MTL memory regions are coactivated during successful retrieval as well as during successful encoding of emotional memories (Dolcos et al., 2004b), correlations between retrieval success activity in the amygdala and the memory-related MTL regions were performed for both R and K responses and compared for emotional and neutral pictures. Consistent with the predicted differential effect of emotional arousal on recollection and familiarity, the study generated three main findings. First, the memory-enhancing effect of arousal was driven by recollection-based responses (figure 6.4A). This finding extends the behavioral evidence that emotional arousal differentially enhances recollection (Ochsner, 2000) by showing that this effect extends over a period of one year. Second, this differential effect of arousal on recollection was accompanied by greater recollection vs. familiarity-based retrieval success activity in both the amygdala and the hippocampus, but not in other MTL memory regions such as the entorhinal cortex (figure 6.4B). These findings provide clear evidence that activity in the amygdala and hippocampus is associated not only to enhanced feelings of Figure 6.4 Effect of arousal on recollection and familiarity: behavioral and fMRI results. (A) The memory enhancing effect of emotion was driven by recollection. Corrected recognition scores (Hits – False Alarms) for emotional (positive and negative collapsed) and neutral pictures are presented. Emo = Emotional; Neu = Neutral; Remember = Recollection-based responses; Know = Familiaritybased responses; FAs = False alarms. (B) Dissociable effect of emotion on retrieval success (RS) activity in the amygdala and hippocampus vs. entorhinal cortex. In the right amygdala and hippocampus head, RS activity for emotional pictures was greater for recollection than for familiarity, whereas in the entorhinal cortex it was similar for both types of retrieval. RS = Hits – Misses; Remember = Recollection-based RS (Hits-R > Misses); Know = Familiarity-based RS (Hits-K > Misses). Functional Neuroimaging of Emotional Memory 123 recollection for emotional stimuli, but also to enhanced successful recollection of emotional stimuli. These results provide the first converging evidence that links findings from behavioural studies showing arousal-mediated enhancement of recollection (Ochsner, 2000) with findings from functional neuroimaging studies associating the amygdala and the hippocampus with emotional arousal (Zald, 2003) and recollection (Yonelinas, 2002), respectively. Finally, correlation analyses showed that the amygdala and the hippocampus were more systematically coactivated during successful recollection of emotional pictures than during successful recollection of neutral pictures (table 6.1). Therefore, these two regions are part of a synergistic mechanism in which emotion enhances recollection and recollection enhances emotion. Emotion may enhance recollection because reinstating the affective context of the original episode is likely to facilitate the recovery of contextual details, such as where, when, and how the original events happened. Conversely, the recollection of the context surrounding an emotional event is likely to augment the emotional arousal elicited by the event during retrieval. In sum, the available evidence concerning the effect of emotion during memory retrieval shows that the memory advantage for emotionally arousing stimuli is associated with activity in both the amygdala and the MTL memory structures, and that the differential effect of arousal on recollection is accompanied by both amygdalar and hippocampal involvement but is independent of processing in the entorhinal cortex. Table 6.1 Correlations (R-scores) between the amygdala and the MTL memory regions. Correlations were calculated between the amygdala and the MTL memory regions showing greater recollectionvs. familiarity-based retrieval success activity for emotional than for neutral pictures. The greatest differences were in the case of recollection-based responses. RS = retrieval success activity (Hits – Misses), rRS = recollection-based RS, kRS = familiarity-based RS. MTL region/ Amygdala Hippocampal region Emotional rRS Neutral rRS Emotional kRS Neutral kRS Head (R) .94**** .97**** .98**** .86* Body (L) .82* .71 .90*** .74 Tail/PPHG (R) .87** .33 .48 .72 Tail/PPHG (L) .77* .66 .81* .76* * p < 0.05; ** p < 0.01; *** p < 0.005; **** p < 0.0005. 124 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza CONCLUDING REMARKS, OPEN ISSUES, AND FUTURE DIRECTIONS The overarching goal of this chapter was to review the functional neuroimaging evidence concerning the effect of emotional arousal on the neural mechanisms underlying the encoding and retrieval of emotional episodic memory in neurologically intact humans. An important contribution in elucidating these mechanisms was made by the introduction of event-related designs that allow comparisons of brain activity for successfully vs. unsuccessfully encoded/ retrieved items (i.e., the subsequent memory paradigm). This paradigm has provided an ideal way of linking the memory-enhancing effect of emotion observed behaviorally to the neural mechanisms underlying this phenomenon. The available ERP and ER-fMRI studies using this paradigm provide evidence for possible mechanisms of the memory-enhancing effect of emotion. ERP evidence from encoding studies suggests that emotional stimuli have privileged access to processing resources, and this effect may be linked to increased successful encoding activity for emotional stimuli in specific MTL and PFC regions. Evidence from ER-fMRI studies of encoding also shows that the memoryenhancing effect of emotional arousal is associated with the coactivation of two apparently independent MTL systems: an amygdalar system, typically associated with emotional processes, and a hippocampal system, typically associated with mnemonic processes. These findings from neurologically intact humans, along with the findings from animals, provide strong converging evidence for the modulation hypothesis. As for the functional neuroimaging studies of emotional memory retrieval, the available evidence shows that similar MTL mechanisms are also involved during successful retrieval of emotional memories. Finally, studies of retrieval also show that the differential effect of emotional arousal on recollective processes is associated with dissociable patterns of activity in the amygdala and the MTL memory structures. Although functional neuroimaging studies provide valuable evidence concerning the neural mechanisms underlying the beneficial effect of emotion on memory, a number of issues remain unclear. First, one open issue is the role played by emotional valence in the observed effects. While most of the evidence suggests that arousal is the main factor influencing the memoryenhancing effect of emotion, the neural correlates underlying the contribution of emotional valence have been less specified (e.g., see Levine & Pizarro, this volume; Reisberg, this volume). Of particular interest are differences between the neural mechanisms underlying the effect of positive and negative emotions on memory, as positive and negative emotions have evolved to subserve different functions. Thus, it is reasonable to expect that their contribution to the memory-enhancing effect of emotion may be specifically associated with Functional Neuroimaging of Emotional Memory 125 different neural mechanisms, which work in synchrony with the mechanisms underlying the more general effect of emotional arousal. Second, another unsolved problem is whether top-down processes can alter activity in the neural networks responsible for the modulatory effect of emotion on memory. Specifically, to date, studies of emotional memory have typically manipulated the emotional content of the to-be-remembered material (bottom-up control), rather than the perceived emotional content (top-down control). Using methods similar to those involved in a recently emerging body of literature investigating the neural correlates of emotion regulation ( Jackson et al., 2000; Ochsner et al., 2004) to manipulate the perceived nature of emotions, future studies of emotional memory could better delineate the neural mechanisms underlying the beneficial effect of emotion on memory. Third, it is uncertain how the emotional memory mechanisms described in this chapter operate in people suffering from affective disorders, such as depression and post-traumatic stress disorder (PTSD). For instance, it is known that depressed patients ruminate on negative or unpleasant memories, and that PTSD patients suffer from intrusive traumatic memories. These problems may reflect a pathology in how the memory system processes emotional stimuli, either during the encoding or retrieval stages. Therefore, findings from normal populations (e.g., from studies using manipulation of the mechanisms involved in emotional regulation, along with studies investigating the mechanisms involved in memory suppression; Anderson et al., 2004, Ochsner & Gross, 2005) could provide hints to future studies investigating neural alterations associated with such clinical populations. Furthermore, corroboration of findings from studies on normal and clinical populations (e.g., Mayberg et al., 1999; Protopopescu et al., 2005; Rauch et al., 1996; Seminowicz et al., 2004) will contribute to a better understanding of clinical conditions associated with augmented processing and experiencing of emotional memories. Fourth, a very important area that has not received enough attention is how emotional memory mechanisms are affected by healthy and pathological aging. Although emotional processing is generally preserved in healthy older adults, there is evidence that they show a bias towards positive emotions (positivity bias) not seen in younger adults (Mather, this volume; Mather et al., 2004). There is virtually no evidence about whether the age-related positivity bias has an impact on the neural correlates of emotional memory. In contrast with healthy older adults, in patients with Alzheimer’s disease (AD) the memory-enhancing effect or emotion is disrupted (Hamann, Monarch, & Goldstein, 2000; Kensinger et al., 2002). One of the earliest signs of AD is the deterioration of anterior MTL regions, such as entorhinal cortex and the amygdala. Thus, the study of emotional memory in AD is also important from a diagnostic point of view. 126 Florin Dolcos, Kevin S. LaBar, and Roberto Cabeza Fifth, another issue concerns investigation of gender-related differences in the neural mechanisms of emotional memory. For instance, evidence from studies investigating sex-related differences in the neural correlates of emotional memory encoding have identified an interesting hemispheric asymmetry concerning the role of the amygdala, with the left amygdala being more involved during emotional memory encoding in females and right amygdala being more involved during emotional memory encoding in males (Cahill et al., 2004). It is not clear, however, whether a similar pattern of lateralization is also present during retrieval of emotional memories. Sixth, a more recently emerging issue concerns the dissociation between the neural mechanisms of emotional memories for content vs. context (Erk, Martin, & Walter, 2005; Maratos et al., 2001; Medford et al., 2005; Smith et al., 2004b). That is, most of the studies of emotional memory have investigated the neural correlates of encoding/retrieval of items with emotional vs. neutral contents. However, it is not clear whether the same mechanisms are responsible for the enhancement of memory for items embedded in emotional vs. neutral contexts. Future studies investigating this issue will contribute to better understanding of the neural correlates distinguishing the enhancing effect of emotional content vs. context on episodic memory. Seventh, whereas most memory and emotion studies have focused on how emotion enhances memory, very little is known regarding the mechanisms of how emotion impairs or distorts memory. There is some evidence that experiencing extreme emotions can lead to forgetting of traumatic events experienced in childhood (Williams, 1994, but see Goodman & Paz-Alonso, this volume; Kihlstrom, this volume). In addition, as evidence from the eyewitness testimony literature shows, highly arousing emotions may hinder memory for “peripheral” details of emotional events (Christianson, 1992a; Christianson & Engleberg, this volume; Mathews, this volume). Such studies could contribute to better understanding of the neural correlates of affectivecognitive interactions, in general, and of those underlying the modulatory effect of emotion on memory, in particular. Finally, critical to further elucidation of the neural correlates of emotional memory is that, in addition to corroborating converging evidence from separate studies investigating various aspects of the processing (e.g., animal vs. human, normal vs. clinical, ERP vs. fMRI; Cahill et al., 1995; McGaugh, 2004; Dolcos et al., 2002; Dolcos et al., 2004b), future studies should also aim at more extensively investigating different aspects within the same studies (e.g., pharmacological and neuroimaging, lesion and neuroimaging, personality and neuroimaging and genotyping, etc.; Richardson et al., 2004; van Stegeren et al., 2005). In conclusion, research on the neural correlates of the memory-enhancing effect of emotion has contributed substantially to the recent development in Functional Neuroimaging of Emotional Memory 127 the field of cognitive neuroscience of emotion. An important contribution has been from using functional neuroimaging tools that allow investigation of the neurologically intact human brain. Evidence from recent functional neuroimaging studies provides strong evidence for the modulation hypothesis by clearly showing that the memory enhancing effect of emotion is associated with joint activity in both an emotion-based system involving the amygdala and a memory-based system involving the hippocampus and associated MTL memory regions as well as the PFC. Moreover, evidence from recent functional neuroimaging studies of emotional memory retrieval has extended this finding by showing that mechanisms similar to those identified during initial encoding are also involved during successful retrieval of emotional memories. Finally, recent studies of emotional memory retrieval have also revealed the neural mechanisms underlying the differential effect of emotion on recollectionvs. familiarity-based retrieval. Thus, involvement of functional neuroimagimg tools in corroboration with other methods will prove essential in future studies investigating the neural mechanisms of affective and mnemonic processes in both normal and clinical populations.